There are a variety of properties that might be desirable for dyes that are intended for use as markers for detection of proteins or nucleic acid hybridization. These can include the ability to bind to a protein, lipid or nucleic acid, the capability of incorporation into nucleic acids by enzymatic means when attached to a nucleotide, a lack of steric hindrance that could potentially interfere with hybridization, water solubility, lack of aggregation, ability to intercalate into double-stranded nucleic acids and the presence of a reactive group that allows attachment of the dye to a nucleotide or other desirable target. Suitable dyes could have many of these properties but do not need to have them all. For instance, the ability to intercalate may allow detection of hybridization events in the presence of unhybridized probes or it may provide increased hybridization stabilization. Examples of these applications are disclosed in European Patent Application EP 0 231 495, U.S. Pat. Ser. No. 5,994,056 and U.S. Pat. Ser. No. 6,174,670, all of which are incorporated by reference. Similarly, the ability to be incorporated by an enzyme is a useful property when carrying out enzymatic labeling of nucleic acids. Labels that are inhibitory towards incorporation can still be used in some methods where nucleic acids are chemically synthesized rather than using enzymatic means. Also, nucleotides with reactive groups such as allyl-amine may be incorporated enzymatically into nucleic acids and then in a second step they are post-synthetically modified by attachment of dyes. Steric hindrance may be compensated to some degree by the nature of the linker joining the dye to a nucleotide with regard to both the length and the constituents of the linker. For a discussion of this last point, see U.S. patent application Ser. No. 2003/0225247, hereby incorporated by reference.
The particular spectral characteristics of dyes are also important qualities. Although broad-spectrum white light can be used as a source of excitation, lasers with defined set wavelengths are most commonly employed. As such, dyes that would find most immediate use would have excitation wavelengths that can make use of such laser emissions. Emission wavelengths are of a more flexible nature since filters can be used to isolate a particular part of the spectrum. However, it should be noted that there are a number of machines used for detection of labeled nucleic acids that have been designed with dyes that are commonly used. For instance, there are a number of slide scanners that have been optimized for detection of nucleic acids labeled with the Cy3 and Cy5 dyes described by Waggoner et al. in U.S. Pat. Ser. No. 5,268,486 (incorporated herein by reference). On the other hand, the availability of dyes that have useful properties but have wavelengths that are not commonly used can prove to be an incentive to adopt lasers with compatible wavelengths.
A set of dyes with well separated emission spectra may find use where more than one fluorophor is to be used at the same time. Well known applications in this are immunostaining for various proteins in cells, in situ hybridization for multiple targets, non-radioactive sequencing, nucleic acid array analysis, protein array analysis, as well as non-specific cellular staining with dyes having general affinities for proteins or lipids. On the other hand, overlapping spectral characteristics also have applications; for instance, emission by one fluorophor may be used to excite a second fluorophor through energy transfer when distances are sufficiently close.
Among the dyes that have been most widely used as markers for proteins and nucleic acid labeling are members of the xanthene, coumarin, cyanine and asymmetric cyanine dye families. Xanthene dyes are among the earliest dyes used for biological staining, where fluorescein was used to work out many of the techniques for labeling proteins and nucleic acids. The basic structure of fluorescein molecules can be depicted as:

Related xanthene compounds that have also been used as labels include rhodols and rhodamines. Their basic structure is as follows:

The R group attached to the central structure is typically a substituted phenyl group although as described in U.S. patent application Ser. No. 2003/0225247 (hereby incorporated by reference), aphenylic versions are also suitable as dyes.
Another family of dyes that have enjoyed widespread use is based upon derivatives of coumarin. The basic structure of coumarin is as follows:

Typically, coumarin derivatives will be dyes when R is an OH or an amine group. Useful compounds have also been made where R is further modified such that an enzymatic cleavage event converts the R group into an OH or amine group. Thus this proto-dye or dye precursor can be used as marker for the presence of an enzyme that is capable of converting a coumarin compound into a fluorescent dye. Discussions of such methods are disclosed in U.S. Pat. Ser. No. 5,696,157 and U.S. Pat. Ser. No. 5,830,912, both of which are incorporated by reference.
As described above, a large number of useful dyes are based upon cyanine dyes. The basic structure of Cyanine dyes is as follows
As will be discussed later, major factors in the particular spectral qualities of these dyes is dependent upon the number “n”, the nature of “X” and “Y” and functional groups that extend the aromaticity of the dyes.
Other compounds that were functionally considered to be Cyanine-type dyes (see U.S. Pat. Ser. No. 5,268,486 hereby incorporated by reference) are the merocyanine and styryl dyes whose structures are:

There are a variety of atoms that have been used in the X and Y positions. These have included carbon, sulfur, oxygen, nitrogen and selenium. When X or Y is a carbon, this portion of the dye is an indolinium moiety. When X or Y is substituted by sulfur, oxygen or nitrogen this portion is respectively described as a benzothiazolium, benzoxazolium or a benzimidazolium moiety.
Another version of styryl dyes can have picoline or quinoline moieties instead of the benzazolium group, thereby having the structures:

Asymmetric cyanine dyes contain one portion that is essentially the benzazolium portion of the cyanine dye family but connected to this portion by the methine bridge is a different aromatic compound. Their structure is as follows:
The aromatic moiety can be a six membered aromatic or heteroaromatic ring
Improvements to these dyes have been carried out by substitution of various groups onto the basic structure, i.e. on the carbons and nitrogens of the preceding structures or where H or R groups are featured. Additionally, other rings may be fused to various parts of the rings in the structures above, thereby generating more complex structures. These modifications have led to shifts in the excitation and emission characteristics of the dyes that allow a large number of dyes with same basic structure but having different spectral characteristics, i.e. modifications can be made in their structure that can alter the particular wavelengths where these compounds will absorb and fluoresce light. As described above, the cyanine dyes can have a general structure comprising two benzazolium-based rings connected by a series of conjugated double bonds. The dyes are classified by the number (n) of central double bonds connecting the two ring structures; monocarbocyanine or trimethinecarbocyanine when n=1; dicarbocyanine or pentamethinecarbocyanine when n=2; and tricarbocyanine or heptamethinecarbocyanine when n=3. The spectral characteristics of the cyanine dyes have been observed to follow specific empirical rules. For example, each additional conjugated double bond between the rings usually raise the absorption and emission maximum about 100 nm. Thus, when a compound with n=1 has a maximum absorption of approximately 550 nm, equivalent compounds with n=2 and n=3 can have maximum absorptions of 650 nm and 750 nm respectively. Addition of aromatic groups to the sides of the molecules has lesser effects and may shift the absorption by 15 nm to a longer wavelength. The groups comprising the indolenine ring can also contribute to the absorption and emission characteristics. Using the values obtained with gem-dimethyl group as a reference point, oxygen substituted in the ring for the gem-dimethyl group can decrease the absorption and emission maxima by approximately 50 nm. In contrast, substitution of sulfur can increase the absorption and emission maxima by about 25 nm. R groups on the aromatic rings such as alkyl, alkyl-sulfonate and alkyl-carboxylate usually have little effect on the absorption and emission maxima of the cyanine dyes (U.S. Pat. Ser. No. 6,110,630, hereby incorporated by reference).
As described above, alteration of spectral qualities is only one useful modification that can be made to a dye. In another instance, modification of a dye by a sulfonate group may increase the stability of many dyes and thereby resist “bleaching” after illumination. Modification of dyes by sulfonation was later applied in the modification of cyanine dyes with reactive groups (U.S. Pat. Ser. No. 5,569,766 hereby incorporated by reference), where it was reported that the sulfonation decreases aggregation of labeled materials. It was further applied to xanthenes, coumarins and the non-benzazolium portion of asymmetric cyanine dyes (U.S. Pat. Ser. No. 5,436,134, U.S. Pat. Ser. No. 6,130,101 and U.S. Pat. Ser. No. 5,696,157, all of which are hereby incorporated by reference). Modifications of dyes haves also been made to increase their affinity or selectivity towards binding to nucleic acids (European Patent Application Serial No. EP0 231495, U.S. patent application Ser. No. 2003/0225247 and U.S. Pat. Ser. No. 5,658,751, all of which are incorporated by reference).
In many cases, the utility of these dyes has been achieved by synthesis of compounds with a reactive group that allows attachment of the dye to a target molecule. For instance, although cyanine dyes in themselves had been known for many years, it was only when derivatives were described with reactive groups (U.S. Pat. Ser. No. 5,268,486 hereby incorporated by reference) that they found widespread use in labeling proteins and nucleic acids. Their versatility was then increased by disclosure of other groups that could be used to attach cyanine dyes to suitable partners (U.S. Pat. Ser. No. 6,114,350 and U.S. patent application Ser. No. 2003/0225247, both of which are hereby incorporated by reference). An exemplarary list of electrophilic groups and corresponding nucleophilic groups that can be used for these purposes are given in Table 1 of U.S. Pat. Ser. No. 6,348,596 (hereby incorporated by reference).
A variety of linker arms may be used to attach dyes to targets. Commonly used constituents for linkers are chains that contain varying amounts of carbon, nitrogen, oxygen and sulfur. Examples of linkers using some of these combinations are given in U.S. Pat. Ser. No. 4,707,440, hereby incorporated by reference. Bonds joining together the constituents can be simple carbon-carbon bonds or they may be acyl bonds (U.S. Pat. Ser. No. 5,047,519), sulfonamide moieties (U.S. Pat. Ser. No. 6,448,008) and polar groups (U.S. patent application Ser. No. 2003/0225247) all of which are hereby incorporated by reference.